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Note' I2p. 

.MEASUREMENT TECHNIQUES PREDICTIVE MEASUREMENT, .SOCIAL 

SYSTEMS 

This paper presents a new approach to the measurement of change within human 
systems (individuals, groups, organizations, communities). It is reported as being a more 
realistic and accurate approach than the use of simple raw-score change, easier to 
calculate than the analysis of co-variance and other procedures designed to equalize 
the effects of individual difference in pre-scores, and more able to provide meanirigtuj 
comparisons between cases. The approach is called percentage gam and lo 
computed by taking the percentage that the actual raw score point change is, of the 
number of points available between the pre-score and the highest possible score^ 
Percentage gain scores were calculated for a sample of 196 workshop students and 
compared with simple raw change scores in several statistical analyses. (Author) 
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This paper presents a now approach to the measurement of 
change within human systems (individuals, groups, organi- 
zations, communities). It is reported as being a more 
realistic and accurate ax:-proach than the use of simple 
raw-score change, easier to calculate than the analysis 
of CO- variance and other procedures designed to ecxualize 
the effects of individual differences in pre-scores, and 
more able to provide meaningful comparisons between cases 
The approach is called percentage g ain and is computed by 
taking the percentage that the actual _ raw score point 
change is of the n’omber of points available between tne 
pre-score and the liighest possible score. percentage 
gain scores were calculated for a sample of 196 workshop 
students and compared wii..h simple raw change scores in 
several statistical analyses. 



Introdu ct ion to the Problem 

Researchers, who have attempted to compare the amounts of change 
achieved by a number of persons along a particular dimension over 
a period of time, are v;ell aware of the fact that individual dif- 
ferences in starting positions confuse the meaning and contaminate 
the validity and fairness of the comparisons. The problem is one 
of sufficient magnitude to stop researchers in the tracks of their 



own research and draw them into th ■ quest for more accurate, de- 
fensible and less complex methods than seem currently available 
for managing the measurement and comparison of individual growth. 

■! Thorndike3 initially called attention to this problem in 1924 
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Diederichl carefully summarisjod the issues involved in 1956, and 

Harris^ organized a scholarly review of the problem in a 1963 te^>.t. 
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A major concern is that, in the more customary methods of analysis 
of pre- and post-scores on achievement- type inventories, the larg- 
est average gains are made by lov; starters and the smallest average 
gains are made by high starters. The phenomenon of the "regression 
of post-scores tov/ard the mean" is cited as one causative factor, 
and the greater difficulty facing the high pre-scorer as compared 
to the low pre-scorer is cited as another. 

Essentially, the question behind the problem is; When measur- 
ing any kind of change in a person, where there is a pre- and a 
post-assessment, how can the change be calculated so that the in- 
dividuality of the person’s pre-score is accounted for and yet his 
change can be represented in terms which allow for a fair compari- 
son to changes in others? 

Several statistical procedures for minimizing or eliminating 
the impact of pre-score differences on the measurement and compari- 
son of change have been developed. Examples of these are: analy- 
sis of co-variance; regression analysis; arranging pre-scores into 
separate populations of high, middle and low and then analyzing 
change within these separate populations; initial testing for sig- 
nificance of difference between pre-scores to establish possible 
non-difference in starting positions. Some researchers, finding 
these procedures either too laborious or complex for the goodies s 

of the consequences or too inappropriate for their own purposes, 

* 

choose not to deal v/ith the issue of pre-score contamination at 
all. Instead, they may settle for simple gain in raw score points 
as their measure of change. 
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This article suggests one more approach to the measurement of 
individual change. It is a non-complex procedure which employs the 
post-score in relation to the distance between the p -e-score and 
the highest possible score . It appears to yield a more meaningful 
assessment of individual change and allows for a more accurate com- 
parison of changes made by different persons. This statistic, 
which the authors call Percentage Gain , responds to such questions 
as: Whore the highest possible score is 100 , did the person who 

began at 13 and then gained ten points achieve an equal, greater, 
or lessor amount than the person who began at 2£ and then gained 
ten points? Did the person who began at 2 ^ and then gained eighty 
points achieve an equal, greater or lesser amount than the person 
v;ho began at 8^ and then gained tv.’enty points? What about the per- 
son who began at and gained thirty points as compared to the 
person who began at 5^ and gained twenty-five points? Employing 
Percentage Gain as the statistic, the answer to the first question 
is “lesser, " and the answers to both the second and third questions 
are "equal." This is so, since change , here, is a measure of the 
degree to which each person moved toward complete or 100% gain, 
considering each person's own individual initial score and the per- 
centage of the distance his post-score covered between his initial 
score and the highest possd.ble score. Each person has, in a sense, 
his own "track to run," and the size of the units of gain are not 
the same for everyone but are determined by the person's own start- 
ing position. Thus, the person who entered with a score of 13 and 
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gained ten points covered only 11.7% of his remaining eighty-five 
points. The person who entered with a score of 7 0 and gained ten 
points covered 33.3% of his remaining thirty points. 

Whether or not the use of Percentage Gain actually yields 
findings which are significantly different from simple gain scores 
is considered in t le next section of this paper. The data used to 
compare these two methods fo- measuring change were collected from 
196 adults immediately before and after their attendance at a six- 
day social science summer workshop. The same instrument, request- 
ing the subjects to respond +.0 twenty positional statements and 
having a lowest possible score of 2^ and a highest possible score 
of 160 , was administered before and after the workshop. 

While this article focuses on the measurement of change w’ithin 
individual persons, Percentage Gain has also been employed by the 
authors to assess change within larger human systems, e.g., groups, 
organizations and communities. 

Statistical Investigation of 
Percentage Gain 

When the final score for an individual exceeded his initial 
score, the percentage gain score was defined as: 




where X 2 = final score 

== initial score 

R = highest possible score minus - 



(maximum possible gain) 



initial score 
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When the final score was less than the initial score, percent- 
age gain was defined as: 



P.G. 





""n / 



100 



where = final score 

= initial score 

R = initial score minus lowest possible score 
^ (maximum possible loss) 

a 

In the immediately above case, /percentage gain score would be nega- 
tive. The total range for percentage gain in both negative and 
positive directions would be -100 to +100. A frequency distribu- 
tion of these scores would be expected to exhibit negative skewness 

in 

since the majority of changes would be/a positive direction. 

Pre- and post-test scores were employed to compute (a) percent- 
age gain scores, and (b) simple change scores for each of the 196 
students in the workshop. Simple change was defined as the actual 
difference between post- and pre-test scores. 

That percentage gain scores do differ from simple change scores 
was evident upon examination of the data. For instance, nine of 
the workshop students had simple change scores of 15 points. The 
percentage gain scores for the same nine students ranged from 52 
to 12, specific scores being 52, 31, 29, 26, 25, 23, 22, 18, 12. 
Since percentage gain scores are defined as per cent of possible 
change, higher per cent change was associated with higher initial 
score. Thus, the student with a percentage gain score of 52 had a 
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pre-tGst. score of 146, and the student with a percentage gain score 

of 12 had a pre-test score of 110. 

Further evidence of the difference between the two types of 
scores is provided when all the percentage gain scores are rank- 
ordered and all the simple change scores are rank-ordered, the two 
ranks for each score are listed side— by— side, and then the differ- 
ences between all ihe paired ranks are calculated and laid out for 
observation. Table 1 presents a suiranary of how much and how many 
of the 196 paired ranks differed from each other. As indicated, a 
person's achievement in comparison to the ' achievements of others 
can vary greatly depending on whether the achievement is measured 

by percentage gain or simple change. 

TABLE 1 

DIFFERENCES BETt'JEEN THE RANKS OF PAIRED SIMPLE 
CHANGE AND PERCENTAGE GAIN SCORES 



Sizes of Differ- 
ences Between 
Paired Ranks 


No', of Paired 
Ranks Having 
that 

Difference 


% of Paired Ranks 
Having that Dif- 
ference 


0 


26 


13% 


1-2 


35 


18% 


3-7 


34 


17% 


8-12 


30 


15% 


13-19 


32 


16% 


20-26 


20 


10% 


28-41 


12 


7% 


44-68 


7 

N = 196- 


4% 

100% 



The transformed scores were compared using several statisti- 
cal analyses. Table 2 presents rank order correlation coefficients 
for percentage gain and simple change scores. Although, as already 
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ind.ic 0 ted, there were many discrepencies in percentage gain scores 
among individuals with the same simple change, a surprisingly high 
rank order correlation of +.953 was found for the total group. A 
partial explanation of the high correlation might be found in the 
fact that positive scores on one variable were always associated 
with positive scores on the other, the same perfect relationship 
holding for negative scores. 

To examine more closely the nature of this relationship, the 
rank orders of the simple change scores were split into two groups 
of high changers and low changers. Then correlations with ranks 
of the assocj ited percentage gain scores were calculated. As in- 
dicated in Table 2, the correlations were still very high within 
the two groups — +.956 for the high changers and +.902 for the low 
changers. The nature of this relationship was then examined even 
more closely by splitting the rank orders of the simple change 
scores into four groups of highest to lowest changers and then com- 
paring them to the ranks of the associated percentage gain scores. 
This latter arrangement, as also indicated in Table 2, showed some 
differences between the sizes of the correlations within the four 
groups. The highest correlation (+.988) actually occurs within 
the highest changer group, the next highest correlations (+.659 
and +.675 occur within the two middle changer groups, and the low- 
est correlation (+.540) occurs within the lowest changer group. 
While all correlations were statistically significant ( ^ .01), 
they decrees ingly accounted for all factors involved — from 97 . 6 % 
(.988 X .988) to 29.2% (.540 x .540). 
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TABLE 2 

RANK ORDER CORRELATIONS BETWEEN SIMPLE CHANGE 
SCORES AND ASSOCIATED PERCENTAGE GAIN SCORES 



Change Level and Size 


of Population 


Rank Order Correlation 


Total (196) 




.953 


High Changers 


(98) 


.956 


Low Changers 


(98) 


.902 


Highest Changers 


(49) 


.988 


Middle High Changers 


(49) 


. 659 


Middle Low Changers 


(49) 


. 675 


Lowest Changers 


(49) 


. 540 



Table 3 presents correlation coefficients whirb indicate the 
nature of the relationships of percentage gain and simple change 
scores to pre- and post-test scores. 



TABLE 3 

CORRELATIONS BETWEEN PERCENTAGE GAIN, SIMPLE 
CHANGE AND PRE- AND POST-TEST SCORES (N = 196) 



Variable 


Pre-Test 


Post-Test 


Percentage Gain 


-.39 


.63 


Simple Change 


-.61 


.46 



Both percentage gain and simple change scores were associated 
negatively with pre-test scores and positively with post-test scores. 
However, the relationship with post-test scores was somewhat higher 
for percentage gain than for simple change. Simple change tended 
to exhibit relatively higher relationships with pre-test scores. 
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The means and standard deviations of the percentage gain 
scores and the simple chcinge scores are given in Table 4. Also 
given are the coefficients of variation obtained by the formula: 
C.V, = cr/^{iQ0), Both mean and standard deviation were larger 
for the percentage gain than for simple change scores. Because of 
the difficulty of comparing standard deviations based upon differ- 
ent units of measurement, the coefficients of variati.on also are 
reported. The coefficient tends to place different standard de- 
viations on a similar scale. The computed coefficient of variation 
was less than 100 for percentage gain scores, but above 100 for 
simple change scores. Thus, it appears that the range of varia- 
tion between scores v;ithin a distribution of percentage gains is 
smaller than within a distribution of simple changes. 

TABLE 4 

MEANvS, STANDARD DEVIATIONS AND COEFFICIENTS OF 
VARIATION FOR PERCENTAGE GAIN AND 
SIMPLE CHANGE SCORES (N = 196) 



Variable 


Mean 


Sigma 


C.V. 


Percentage Gain 


26.06 


21.80 


84 


Simple Change 


14.80 


16.10 


109 



Skewness and kurtosis of the percentage gain and simple change 
scores were investigated using Fisher's k statistics. The result- 
ing statistics may be referred to the standard normal curve. 

Values are shown in Table 5. 
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TABLE 5 

SKEWNESS AND KURTOSIS COEFFICIENTS FOR PERCENTAGE 
GAIN AND CHANGE SCORES (N = 196) 



Variable 




Values 


P * 


Percentage Gain: 


Skewness 

Kurtosis 


-1.50 

.02 


N.S. 

N.S. 


Simple Change: 


Skewness 

Kurtosis 


-2.68 

2.17 


.05 

.05 



As may L© antic ipa.t©d^ negative skewness was found for Lotli 
percentage gain and simple change scores. For simple change scores 
the skewness was statistically significant, as was the kurtosis, 
the distribution being flat-topped. There is some suggestion that 
the percentage gain transformation has a normalizing influence. 

The 196 workshop students were subdivided into two groups 
based on age categories: (a) under 40; and (b) over 40. The ob- 

ject was to investigate the extent to which percentage gain and 
simple change t"' ; res would differentiate the age groups in test.- 
of significance. Table 6 presents t-tatios for mean differences 

between che groups . 

As indicated in Table 6, the percentage gain scores produced 
a significant mean difference, whereas the simple change scores 
did not, the t-ratio of the percentage gain mean difference being 
significant at the .05 level of probability. Thus, in this par 
ticular case an investigator would have been led to different con- 
clusions on the null hypothesis (equal means) depending upon the 
manner in which the measures were computed. 
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TABLE 6 

MEANS AND t-RATIOS FOR TWO AGE GROUPS ON PERCENTAGE 
GAIN AND SIMPLE CHANGE SCORES (N = 196) 



Variable 


Statistic 


Age Group a 
20-39 yrs. 
(N = 105) 


Age Group b 
40-50+ yrs. 
(N = 91) 


P 


Percentage 


Mean 


30.12 


23.05 




Gain 


t-Ratio 


2.26 


.05 


Simple 


Mean 


17.19 


12.99 




Change 


t-Ratio 


1.80 


N.S. 



Conclusion 



In all, percentage gain represents another alternative avail- 
able to researchers when they are attempting to assess growth and 
change in human systems. Based upon the analysis of the data in 
this study, the following observations can be made regarding the 
behavior of the percentage gain scores. 

1. Persons achieving the very same raw score gains varied widely 
in their percentage gain scores. Thus, when a set of scores 
were rank-ordered on a percentage gain basis, they assumed dif- 
ferent positions than when they were rank ordered on a simple 
change basis. 

2. Percentage gain scores were more highly correlated with post- 
.test scores, and simple change scores were more highly corre- 
lated with pre-test scores. 

3. The range of variation between scores within a distribution of 
percentage gains was smaller than within a distribution of sim- 
ple changes . 

4. The use of percentage gain appeared to have a normalizing ef- 
fect upon a distribution of scores. 

5. While the use of simple change may penalize the high starters 
by standardly reducing their gains belov; that of the low start- 
ers, the use of percentage gain does not. 
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6. Analysis of data by percentage gain and simple change can pro- 
duce different interpretations of that data. 
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